I’m curious what the motivation is for this?

On Apr 17, 2015, at 3:13 PM, Roman Sutormin <notifications@github.commailto:notifications@github.com> wrote:

You can view, comment on, or merge this pull request online at:

#13

Commit Summary

• Changes in perl and python libs supporting calls to auth service.

File Changes

• M Bio-KBase-Auth/lib/Bio/KBase/AuthToken.pmhttps://github.com/Changes in perl and python libs supporting calls to auth service. #13/files#diff-0 (51)
• M python-libs/biokbase/nexus/client.pyhttps://github.com/Changes in perl and python libs supporting calls to auth service. #13/files#diff-1 (4)
• M python-libs/biokbase/nexus/token_utils.pyhttps://github.com/Changes in perl and python libs supporting calls to auth service. #13/files#diff-2 (17)

Patch Links:

• https://github.com/kbase/auth/pull/13.patch
• https://github.com/kbase/auth/pull/13.diff

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13.

Well, this PR is more for discussion rather than for acceptance (at least for a while and I'm sure there will be some improvements in these changes). The idea is to support optional way to use kbase auth service for token extraction and validation in generated JSON-RPC client/server code (it's about python and perl because in java it's the only way and on javascript client side there is only token supported and no validation at all).

Does this mean you are contemplating not actually using the tokens for authentication? Or is this mixing in a notion of authorization along with the authentication? If it is the latter I think the authorization questions need to be made explicit and outside the token management classes.

On Apr 17, 2015, at 3:36 PM, Roman Sutormin <notifications@github.commailto:notifications@github.com> wrote:

Well, this PR is more for discussion rather than for acceptance (at least for a while and I'm sure there will be some improvements in these changes). The idea is to support optional way to use kbase auth service for token extraction and validation in generated JSON-RPC client/server code (it's about python and perl because in java it's the only way and on javascript client side there is only token supported and no validation at all).

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13#issuecomment-94069774.

Why do we need to use the Globus token for service calls?

I don't think there is any reference to authorization in my PR. Authorization in one or another way should be checked in implementation of server functions (like checking permission for particular user to delete particular shock node or workspace object or to run more than some limited number of jobs on cluster). We discuss here common generated server/client code maintaining authentication only.

What else would you use to contact a service that requires authentication?

Unless there has been a change in policy, the KB project has been using Globus tokens as the means to authenticate service requests for those services that require authentication.

On Apr 17, 2015, at 4:19 PM, Erik Pearson <notifications@github.commailto:notifications@github.com> wrote:

Why do we need to use the Globus token for service calls?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13#issuecomment-94077624.

You just mentioned auth, which is ambiguous, and I wanted to make sure we were on the same page. I agree that authorization is outside scope here.

—bob

On Apr 17, 2015, at 4:39 PM, Roman Sutormin <notifications@github.commailto:notifications@github.com> wrote:

I don't think there is any reference to authorization in my PR. Authorization in one or another way should be checked in implementation of server functions (like checking permission for particular user to delete particular shock node or workspace object or to run more than some limited number of jobs on cluster). We discuss here common generated server/client code maintaining authentication only.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13#issuecomment-94080547.

I know auth_service stands for authorization. But it has "Login" function which is authentication function. This is the only function I use in my changes. It forwards call to GlobusOnline so tokens which it deals with are pure GO ones. The goal I'm trying to achieve is to have some optional way to configure auth end-point in clients and servers to point to fake authentication service in tests. This is getting more important now since we decided to use Travis CI for unit tests and we can not store real password of real test user in git repo.

What else would you use to contact a service that requires authentication?

A token created and manage by KBase? What we need is a secure bit of state that can be used within the kbase framework which provides the identity of the bearer and for which we control the structure, api, policy, etc.

Not being flip -- after we obtain a Globus token, validate it, and use it to obtain the user identity, we should create our own session token, with our own security, expiration, and whatever state we need it to carry. By incorporating the globus token as "our own" we have taken on dependency to the internal implementation details of their token.

The kbase auth service should present this simply as an api, with the token opaque, so that these types of issues can be resolved in isolated code locations.

Unless there has been a change in policy, the KB project has been using Globus tokens as the means to authenticate service requests for those services that require authentication.

What I'm also getting at is that the Globus token provides authorization for certain operations with Globus, one of which is obtaining the user identity (I know they also throw that into the token and we can use it.) But that token does not explicitly provide any such privileges for kbase. We turn that user identity around and then we have, by other means, the ability to do things on behalf of that user within kbase. The fact that we carry the Globus token around as authentication seems more like a convenience than anything else, and that seems to be wearing thin!

The goal I'm trying to achieve is to have some optional way to configure auth
end-point in clients and servers to point to fake authentication service in tests. This is getting
more important now since we decided to use Travis CI for unit tests and we can not
store real password of real test user in git repo.

Perhaps the latter is part of the problem. By definition if we are testing authentication, there will be private credentials passed, private keys stored for signatures, etc. Couldn't configuration of the test suite be maintained privately?

Couldn't configuration of the test suite be maintained privately?
I don't see how it could be possible in Travis CI or any other public test
framework with pull request test support. When somebody makes PR then
Travis CI automatically runs tests for this PR to make sure this pull
request doesn't break anything before owner of repo can merge it. It
happens publicly and if private credentials are in the system then it's not
so hard to make PR printing them to test output.

On Fri, Apr 17, 2015 at 3:22 PM, Erik Pearson notifications@github.com
wrote:

The goal I'm trying to achieve is to have some optional way to configure
auth
end-point in clients and servers to point to fake authentication service
in tests. This is getting
more important now since we decided to use Travis CI for unit tests and we
can not
store real password of real test user in git repo.

Perhaps the latter is part of the problem. By definition if we are testing
authentication, there will be private credentials passed, private keys
stored for signatures, etc. Couldn't configuration of the test suite be
maintained privately?

—
Reply to this email directly or view it on GitHub
#13 (comment).

Right, I was not considering that the tests need to run in the open as well.
Well, if all services are being mocked, then the tests should be able to populate the mock auth service with a user, then issue logins, generate tokens, etc. which will only work in the test environment. The usernames, passwords, keys, etc. can be generated and installed on the fly.
On the back burner I'm testing globus oauth flows, and part of this involves mocking just this stuff -- login, registration, signed tokens, etc. The trick, and why I'm sensitive to binding kbase to globus token internals, is that to simulate it down to a "T" is unpleasant reverse engineering :(
I'm wondering -- aren't there solutions for private stores of configuration information for CI?

I don't understand why keeping test account usernames and passwords in github is a problem. No private data will be stored in a test user account, nor will the test user account be given any private data. Reengineering everything and staging a copy of every service to link to a "fake" auth service just because we're protecting the credentials of the test account sounds very extreme.

Could someone steal the test user account information and use it to post bad things to KBase or do a DOS? Sure... I suppose. Could the very same person set up a 100 new KBase accounts in 60 seconds and do exactly the same thing a hundred times worse? Uh, yes. They could. So what is it that we are working so hard to protect? It's like locking up a water fountain sitting right next to a completely unlocked fire hose.

In fact, permitting the remapping of our services to an arbitrary "fake auth" service screams "back door" to me. It's a back door that could possibly be used to allow an unauthorized user to access private user data... so we are adding new holes to the system and increasing complexity, all because we don't want to expose useless test account passwords in git. I must question the logic here.

Well, Chris, I don't even know should I answer you or not because your points are obviously too far away from normal development practices.
First let me explain problem with public passwords. In this case anybody will be able to login to Globus using public password and change this password. This user will not be accessible from tests anymore. You have to recreate new one and recommit all places in all repos where this user was used. It seems to be much easier to create new test account every time you run tests rather than fix it every time test user was stolen.
Second topic is mocking some parts of your project in order to test each independent block (it's called unit testing). Suppose that you need to test some block of your system relying on workspace service (more precisely on two functions of WS which are save_objects and get_objects). What would average developer do in this case? You create mockup for these two functions and deal with it from tested code as if it was real workspace service. Yes, you can call it fake workspace service if you prefer. What is changing if in your system there are some ways to "remap code to an arbitrary fake workspace service" in test mode? Does it open any back door? The answer is no because it works in test mode only. In production it will be never used. If you're afraid of it then how else would you test elements of your system? If you have alternative secret way to do it without mock ups then please feel free to share it.

Other than testing, I believe there are some opportunities here for a more robust and secure system.
For one, many of the services may be mocked now, but there are opportunities to make the services more modular, so that building, configuring, deploying, and populating a service can be done automatically and quickly. Mocking a service also leads to a deeper understanding of some aspects, such as communication or adherence to design or standards specs. Having to re-use kbase outside of the usual curated box also reveals weaknesses such as hard-coded configuration, difficult builds and deploys, inadequate documentation, and non-standard interfaces.
(Regarding mocking, I'm not 100% convinced it is a safe way to thoroughly test; it certainly is the only way to develop unless the service dependencies can be deployed locally. Take auth -- we have one honking dependency on Globus. An N of 1. To do any local or otherwise disconnected testing which incorporates authentication or identity, some aspects need to be mocked. But it is not always easy to create a mock which operates close enough to the real thing. The API might be close, but there are many particularities of how a service reacts which, in order to mock, would require recreating the entire service!)
I think we also need to be moving further away from the gaping security holes we have. Lack of sign up moderation is just one. Testing against production systems another. CI is pushing us towards having a completely deployable kbase system, isolated from production. This is an opportunity to make kbase fundamentally more secure and robust.
We also need to keep in mind that users are trusting us with their authentication credentials, their private information, and their reputation. There are many ways that an innocent, free-to-create account can be leveraged to not only steal the identity of the user, but of other users of the system, or other users who have a closer relationship with that user (e.g. sharing). As a service that is to be built on user experience and user relationships, I think we have to make them feel very safe, and moreover take the trust they place in us very seriously.

It might be good to consider the implications of not contracting auth to a third party. Perhaps someone could comment on what the Department of Energy might require for us to maintain internally authentication credentials, their private information, and their reputation…, and that if being a US citizen or foreign citizen. I know it can be done, it’s just do we know what is involved?

We opted for buy over build to jump start the project. Now might be a good time to revisit that decision.

On Apr 18, 2015, at 12:22 PM, Erik Pearson notifications@github.com wrote:

Other than testing, I believe there are some opportunities here for a more robust and secure system.
For one, many of the services may be mocked now, but there are opportunities to make the services more modular, so that building, configuring, deploying, and populating a service can be done automatically and quickly. Mocking a service also leads to a deeper understanding of some aspects, such as communication or adherence to design or standards specs. Having to re-use kbase outside of the usual curated box also reveals weaknesses such as hard-coded configuration, difficult builds and deploys, inadequate documentation, and non-standard interfaces.
(Regarding mocking, I'm not 100% convinced it is a safe way to thoroughly test; it certainly is the only way to develop unless the service dependencies can be deployed locally. Take auth -- we have one honking dependency on Globus. An N of 1. To do any local or otherwise disconnected testing which incorporates authentication or identity, some aspects need to be mocked. But it is not always easy to create a mock which operates close enough to the real thing. The API might be close, but there are many particularities of how a service reacts which, in order to mock, would require recreating the entire service!)
I think we also need to be moving further away from the gaping security holes we have. Lack of sign up moderation is just one. Testing against production systems another. CI is pushing us towards having a completely deployable kbase system, isolated from production. This is an opportunity to make kbase fundamentally more secure and robust.
We also need to keep in mind that users are trusting us with their authentication credentials, their private information, and their reputation. There are many ways that an innocent, free-to-create account can be leveraged to not only steal the identity of the user, but of other users of the system, or other users who have a closer relationship with that user (e.g. sharing). As a service that is to be built on user experience and user relationships, I think we have to make them feel very safe, and moreover take the trust they place in us very seriously.

—
Reply to this email directly or view it on GitHub.

In the discussions of security we need to define the threat model. I'm not sure we've every really discussed that in any detail.

From: Thomas Brettin [notifications@github.com]
Sent: Saturday, April 18, 2015 11:53 AM
To: kbase/auth
Cc: Olson, Robert D.
Subject: Re: [auth] Changes in perl and python libs supporting calls to auth service. (#13)

It might be good to consider the implications of not contracting auth to a third party. Perhaps someone could comment on what the Department of Energy might require for us to maintain internally authentication credentials, their private information, and their reputation…, and that if being a US citizen or foreign citizen. I know it can be done, it’s just do we know what is involved?

We opted for buy over build to jump start the project. Now might be a good time to revisit that decision.

On Apr 18, 2015, at 12:22 PM, Erik Pearson notifications@github.com wrote:

Other than testing, I believe there are some opportunities here for a more robust and secure system.
For one, many of the services may be mocked now, but there are opportunities to make the services more modular, so that building, configuring, deploying, and populating a service can be done automatically and quickly. Mocking a service also leads to a deeper understanding of some aspects, such as communication or adherence to design or standards specs. Having to re-use kbase outside of the usual curated box also reveals weaknesses such as hard-coded configuration, difficult builds and deploys, inadequate documentation, and non-standard interfaces.
(Regarding mocking, I'm not 100% convinced it is a safe way to thoroughly test; it certainly is the only way to develop unless the service dependencies can be deployed locally. Take auth -- we have one honking dependency on Globus. An N of 1. To do any local or otherwise disconnected testing which incorporates authentication or identity, some aspects need to be mocked. But it is not always easy to create a mock which operates close enough to the real thing. The API might be close, but there are many particularities of how a service reacts which, in order to mock, would require recreating the entire service!)
I think we also need to be moving further away from the gaping security holes we have. Lack of sign up moderation is just one. Testing against production systems another. CI is pushing us towards having a completely deployable kbase system, isolated from production. This is an opportunity to make kbase fundamentally more secure and robust.
We also need to keep in mind that users are trusting us with their authentication credentials, their private information, and their reputation. There are many ways that an innocent, free-to-create account can be leveraged to not only steal the identity of the user, but of other users of the system, or other users who have a closer relationship with that user (e.g. sharing). As a service that is to be built on user experience and user relationships, I think we have to make them feel very safe, and moreover take the trust they place in us very seriously.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13#issuecomment-94180543.

It might be good to consider the implications of not contracting auth to a third party.

That's an interesting idea, but even if we did our own auth/authz we still need to avoid storing credentials of any kind in Github, and therefore still need a way to run Travis tests without pulling real credentials from Github.

Ok, discussions of security and the threat models could be goals in long term. But it's not so important in short term as test mode in Travis CI. I started this discussion for this testing topic and my suggestion is to focus on closer tasks first. And after that we can start separate discussion about security.

If the entire purpose of this discussion was "how do we test KB code that uses authentication tokens in Travis" it should be laid out like that and I suspect a simple solution is quite doable. Such a solution probably involves the creation of known test tokens, possibly created not by Globus but by a test authority that we configure. Part of the definition of the problem involves explicit enumeration of which services need to honor the requests from the testing site.

From: kkellerlbl [notifications@github.com]
Sent: Saturday, April 18, 2015 12:30 PM
To: kbase/auth
Cc: Olson, Robert D.
Subject: Re: [auth] Changes in perl and python libs supporting calls to auth service. (#13)

It might be good to consider the implications of not contracting auth to a third party.

That's an interesting idea, but even if we did our own auth/authz we still need to avoid storing credentials of any kind in Github, and therefore still need a way to run Travis tests without pulling real credentials from Github.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/13#issuecomment-94185623.

Looking at the code, I don't see comments explaining what is being done here or why. If the idea is to make it possible to use an alternate authn in testing mode, there should be comments in the code to that effect. You can't expect people to be able to find and pick through this PR to understand the motivation for the optional url argument.

Wow, Dan, it's first comment to code itself. Good start. I'll add comments
on Monday (especially if we decide this approach is good enough).

On Sat, Apr 18, 2015 at 2:59 PM, Dan Gunter notifications@github.com
wrote:

Looking at the code, I don't see comments explaining what is being done
here or why. If the idea is to make it possible to use an alternate authn
in testing mode, there should be comments in the code to that effect. You
can't expect people to be able to find and pick through this PR to
understand the motivation for the optional url argument.

—
Reply to this email directly or view it on GitHub
#13 (comment).

Ok, Bob, can I ask your opinion about whole approach? Is it right
direction? If yes are you interested in my corresponding changes in type
compiler allowing to configure/setup auth end-point? If not then what's
your alternative?

On Sat, Apr 18, 2015 at 10:39 AM, olsonanl notifications@github.com wrote:

If the entire purpose of this discussion was "how do we test KB code that
uses authentication tokens in Travis" it should be laid out like that and I
suspect a simple solution is quite doable. Such a solution probably
involves the creation of known test tokens, possibly created not by Globus
but by a test authority that we configure. Part of the definition of the
problem involves explicit enumeration of which services need to honor the
requests from the testing site.

---

From: kkellerlbl [notifications@github.com]
Sent: Saturday, April 18, 2015 12:30 PM
To: kbase/auth
Cc: Olson, Robert D.
Subject: Re: [auth] Changes in perl and python libs supporting calls to
auth service. (#13)

It might be good to consider the implications of not contracting auth to a
third party.

That's an interesting idea, but even if we did our own auth/authz we still
need to avoid storing credentials of any kind in Github, and therefore
still need a way to run Travis tests without pulling real credentials from
Github.

—
Reply to this email directly or view it on GitHub<
https://github.com/kbase/auth/pull/13#issuecomment-94185623>.

—
Reply to this email directly or view it on GitHub
#13 (comment).

First, I just want to say I'd like to keep these discussions constructive. Perhaps we disagree on how to proceed, and that is your and my prerogative. But you asked my advise, and I took precious time to offer my input. Respectfully reject my input if you wish, and move on.

That said, to follow up on my original comment, has anyone consulted globus about whether or not they could provide us with some locked-down credentials that could be checked into git-hub without fear of the account being hijacked. Perhaps an account where auth is only accepted from the IPs of the test machines? We surely aren't the first to run into this issue. I personally like the idea of the system being tested completely using the "real" code rather than a fake auth service.

I concede you are probably right that introducing a fake auth service won't harm security, but it still seems it might hide bugs in the real auth that we might want to know about. Just something to consider for what it's worth.

Chris, sorry for critical style of my response. I didn't mean to offend you. I'm trying to be as much constructive as possible simply because I need to solve this problem for myself and it's now blocking me and many other people from migration tests to Travis CI.
When you suggest to have some set of fixed test users it sounds for me like if you had set of test workspaces in production workspace service and use them in tests. It may be even reasonable for integration tests (in case you don't have good infrastructure or if your tests require some real data which you can not replicate). But not for unit tests taking seconds. It's my opinion of course.
Bugs in real auth should be tested in auth repo which is not supposed to be under pull requests from third party developers. In this place I can live with privately configured tests with real passwords.

Chris beat me to it. Please keep the discussion constructive. Be mindful of your tone both intentional or unintentional.

To weigh in on the debate. I think there is value in having testing stubs available at all levels. Having test accounts with limited/no privileges, having test services, and having mock interfaces are all useful at different stages.

Regarding the actual changes, it is unclear to me how the auth_service_url is intended to be set. My only concern (similar to Chris's) is making sure an external user can't override the validation URL for a service. I don't believe this is possible since the service should be instantiating the auth object. But we would need to ensure that something couldn't slip through initial upstream request. It seems like it would make more sense to override the URL as part of the object creation (i.e. in the new method). In my mind, this would seem less dangerous since it would generally occur long before any user requests are being processed.

And as Matt suggested, please add comments and explanation for the purpose of the new code.

Here is second part of the whole picture. It's pull request for type compiler with corresponding changes in perl and python client/server templates. msneddon/typecomp#1

Oh, and this is my fake implementation of auth_service login function: https://github.com/rsutormin/auth/blob/dev-fake_service/fake-service/src/us/kbase/auth/fakesrv/AuthFakeService.java which I used to check that all parts are working. Of course I'll make it better and deployable after we decide that this approach is right way to solve the problem.

Actually that was Dan G., but I agree with his comment.

On Sun, Apr 19, 2015 at 9:50 AM, Shane Canon notifications@github.com
wrote:

Chris beat me to it. Please keep the discussion constructive. Be mindful
of your tone both intentional or unintentional.

To weigh in on the debate. I think there is value in having testing stubs
available at all levels. Having test accounts with limited/no privileges,
having test services, and having mock interfaces are all useful at
different stages.

Regarding the actual changes, it is unclear to me how the auth_service_url
is intended to be set. My only concern (similar to Chris's) is making sure
an external user can't override the validation URL for a service. I don't
believe this is possible since the service should be instantiating the auth
object. But we would need to ensure that something couldn't slip through
initial upstream request. It seems like it would make more sense to
override the URL as part of the object creation (i.e. in the new method).
In my mind, this would seem less dangerous since it would generally occur
long before any user requests are being processed.

And as Matt suggested, please add comments and explanation for the purpose
of the new code.

—
Reply to this email directly or view it on GitHub
#13 (comment).

Just a question. Is it stipulated that all testing needs to happen in travis CI, with each service tested entirely independently? When and where was this decision made and announced to the group?Testing the modeling code independently of any other services is going to be really complex.

I must confess, I'm deeply confused by what we're trying to achieve with all this work on the "fake" auth service. I suppose I've missed some sort of extremely lengthy development discussion somewhere.

So you want to create, and stage a fake authentication server? And this will be deployed in a test environment somewhere? And then test services will be deployed connected to the fake server? And then you'll do many authenticated tests? So I suppose you'll have a fake workspace posted that also connects to the fake auth server? So every test instance will run a fake auth, fake workspace, and test instance of every server? Where is data needed to load all these fake instances going to be stored? How is fake data being loaded into the fake workspace?

Is someone building infrastructure for all of this? Is this plan documented or discussed anywhere?

Is it stipulated that all testing needs to happen in travis CI, with each service tested entirely independently?

It has been proposed as part of the process guide.

https://github.com/kbase/project_guides/blob/master/Process_guide.md#process-continuous-integration-ci-for-unit-tests-on-each-repository

The next section talks about the proposal for integration testing across services.

https://github.com/kbase/project_guides/blob/master/Process_guide.md#process-continuous-integration-ci-for-systems-level-tests

These docs have been up for a few weeks now, and Fernando announced them at least twice to the mailing list (I forget which one, either All or -devel).

OK. Just to quote from the first few lines in the file you've referenced:

"It is important to clarify that this document is at this stage, an early draft meant to elicit discussion across the KBase project. It is meant to be adapted and fine-tuned, until we can all “sign off” on it, and commit to adopting it as our common process."

This was submitted 19 days ago, primarily written by Fernando and edited by two other people. I don't know if the "services lead" has even looked at this, and he devised his own testing strategy that would seem to need to be reconciled with this. It feels like this has moved from "discussion" to "action" before alot of discussion has taken place.

I have attempted to track policy being devised across the project while also getting some work done on the side, and I must confess, I am beginning to drown in policy.

Sorry, I'm not trying to be difficult. I guess I'm saying I'm seeing general policy being put into practice while the plan to implement the policy still seems pretty foggy to me. One could, for example, ask what it means to have all modules tested using Travis CI. So I see a well-articulated policy in the doc that Keith linked (thankyou keith), but has anyone laid out the implementation plan that Roman and Mike (and others) are following to put this into action? This would be helpful for me.

My impression was that the discussion of the project guide documents was supposed to take place in issues and PRs in the repo. Is that accurate?

Again, this is just my impression, which may be incorrect, but I think the idea is that we are trying to solidify our policies, while knowing that in reality we're not quite caught up with those statements. So, for example, we don't currently have implementation details for Travis CI, nor do we have a 100% fully functioning and reliable CI environment, nor do we have real integration tests that can be run against that CI environment. So the process guide is something we want to work towards, not necessarily something that we'll be fully compliant with tomorrow.

But again, I could be wrong, and I hope someone will correct me if I am.

So, yes, I am sure we all agree that general issues with the process should
not be discussed right here. Chris, can you copy your questions about
implementing the Travis CI plan into an existing or new PR on the process
guide itself? I realize this may feel a bit cumbersome, but I don't know a
better way to do it right now. You should also cc: Fernando to make sure he
doesn't miss it.

Dan Gunter

On Mon, Apr 20, 2015 at 8:45 PM, kkellerlbl notifications@github.com
wrote:

My impression was that the discussion of the project guide documents was
supposed to take place in issues and PRs in the repo. Is that accurate?

Reply to this email directly or view it on GitHub
#13 (comment).

Off-topic, but important one. I'm sure we should have collected/arranged internally in KBase best practices, tutorials or even boot camps and trainings dedicated to testing APIs, scientific algorithms, UI of different complexity on different levels. Seriously. It's critical topic. We can't just hope it's going to be organized by itself.

Hi folks, sorry I wasn't watching this repo, so I missed this discussion... A few points:

• when creating a PR, please always include a description (this one has none), that explains the intent as clearly as you can. It will make it much easier to have a productive discussion about the code in question. Here is just one example of a well-described PR for reference: [WIP] Real Time for text editor ipython/ipython#7987.

  For example, it's totally OK to create a PR that you may not even mean for merging, as a way to encourage a discussion, but if that's the case, please indicate so clearly.

  For example, in IPython we title such PRs with [WIP] (for "work in progress", e.g. [WIP] Liveeditor ipython/ipython#8199) or even loudly flag with do not merge, e.g. Typescript experiment ipython/ipython#8037. Such PRs may be early enough that it's not ready for merging but worthy of common discussion, or they may really never be meant to actually be merged, and simply a way to support a discussion with a prototype implementation of an idea. That way, a conversation about a question can be recorded in a single place, everyone can play with the test implementation, etc, even when it's known that the actual code won't be merged.

• @cshenry: I tried to convey on the mailing list how the kbase/project_guides repository would be the place where we document all project guidelines and policies. I assure you that none of this is meant to impose bureaucracy on people's workflow, but simply to make best practices explicit and hopefully easier for everyone to adopt. These are battle-tested, proven approaches to building robust, reliable tools in an open and collaborative fashion: I'm mostly just trying to codify things that I know from experience work very well in multiple other scientific computing projects. I understand that adapting workflows takes some time, and that every project has its own peculiarities. Hence my desire for explicit and open discussion from all project members on these guidelines, so we can adapt them to the specifics of KBase in a deliberate way. For any further discussion of these, I encourage you to hop over to that repo.

• it should be obvious, but I'll repeat the above request for patience, civility and a constructive tone. It's easy to get frustrated and assume ill-intent across the internet, but in most cases, such an approach is only likely to create more problems than it solves. Please, make every attempt possible to give others in the project the benefit of the doubt, to try and clarify misunderstandings before you assume malice, and to build a productive dialog.